Web Survey Bibliography
In interpreting questions, respondents extract meaning from how the information in a questionnaire is shaped, spaced, and shaded. We carried out five experiments using five questions measured on a five point scale to investigate how the effects of visual heuristics affect the answers to survey questions and test whether effective question writing (using numerical and verbal labels) may overrule visual effects. The experiments were fielded in the CentERpanel. This is an Internet-based panel, although there is no need to have a personal computer with an Internet connection. If necessary, equipment is provided by CentERdata (the agency). The recruitment of new panel members is done through a random sample of landline numbers of candidates. The panel is designed to be representative of the Dutch population. We demonstrate that respondents use the “middle means typical” heuristic only in a polar point scale. When numbers 1 to 5 are added to the polar point format and with fully labeled scales, the effect of spacing between response options disappears. In a second experiment we show that respondents are confused when options are presented in an inconsistent order and do not follow the “left and top means first” heuristic. In a third experiment we find that inter-item correlations are higher in a polar point scale when items are presented on a single screen instead of separate screens. The effect gets smaller when numbers 1 to 5 are added to the polar point scale. In a fully labeled scale no effect is found. In a fourth experiment, we found no evidence that respondents use the “up means good” heuristic: the use of a decremental or incremental scale did not result in different answer distributions. In a fifth experiment, we show that respondents use the “like means close” heuristic: the adding of numbers that differ both in sign and value (-2 to 2) and the adding of different shadings of red and green color affects respondents’ answers. The effect of different numbers was apparent in both a polar point and fully labeled format, while the effect of color was only apparent in the polar point format.
Conference Homepage (abstract)
Web survey bibliography - Internet access Panels (431)
- Visual Heuristics and Answer Formats in Rating Scales; 2009; Toepoel, V. Dillman, D. A.
- Representativeness of web surveys to the general public ; 2009; Sonck, N.
- Nonresponse in the Recruitment of an Internet Panel Based on Probability Sampling; 2009; Hoogendoorn, A., Daalmans, J.
- The utility of online panel surveys versus computer-assisted interviews in obtaining substance-use prevalence...; 2009; Spijkerman, R., Knibbe, R., Knoops, K., van de Mheen, D., van den Eijnden, R.
- Internet research differs from research on internet users: some methodological insights into online...; 2009; Lohmann, M., Schmucker, D. J.
- Mobility, Flexibility and Identity - How the use of mobile questionnaires improves the data quality...; 2009; Hellwig, O., Wirth, T.
- The representativeness of the LISS panel ; 2009; Knoef, M., de Vos, K.
- Mobile phone surveys in mixed mode environment; 2009; Vehovar, V.
- Foundations of Quality Project Overview to Support RFPs; 2008; Tom Evans; Efrain Ribeiro; Renee Smith
- The advisory panel on online public opinion survey quality - Final report June 4, 2008; 2008
- Some consequences of survey mode changes in longitudinal surveys; 2008; Dillman, D. A.
- Sample factors that influence data quality; 2008; Gailey, R., Teal, D., Haechrel, E.
- Representativity of web surveys – an illusion?; 2008; Bethlehem, J.
- Recruitment and retention for a consumer panel; 2008; Tortora, R. D.
- Online panel management practices that minimize satisficing behavior; 2008; Weber, S.
- Modeling campaign dynamics on the web in the 2008 National Annenberg Election Study; 2008; Johnston, R.
- Foundation of quality project overview; 2008
- An online panel as a platform for multi-disciplinary research; 2008; Scherpenzeel, A.
- A look at the impact of boredom on the respondent experience; 2008; Puleston, J., Sleep, D.
- A comparison of results from an alcohol survey of a prerecruited Internet panel and the National Epidemiologic...; 2008; Heeren, T., Edwards, E., Dennis, J. M., Rodkin, S., Hingson, R. W., Rosenbloom D. L.
- An evaluation of the weighting procedures for an online access panel survey; 2008; Loosveldt, G., Sonck, N.
- Validity of an On-Line Panel Approach to Citizen Surveys; 2008; Van Ryzin, G. G.
- Analytic implications of panel data quality; 2008; Harlow, B.
- Pavlov revisited; 2008; De Wulf, K., Friedman, M., Borggreve, B.
- Between random samples and online panels; 2008; Crassweller, A., Rogers, J.
- Digital moms; 2008; Wilbur, C. Rimmer, L.
- Can online panels be truly global?; 2008; Shashkin, A. V.
- On the feasibility of collecting biomarker data in an internet panel; 2008; Avendano, M., Mackenbach, J.
- The Role of Cash Incentives in Online Panelist Motivations: Experimental Results on Unit Response and...; 2008; Taylor, E.
- Internet Access Panels and Public Opinion and Attitude Estimates; 2008; Piekarski, L., Galin, M., Baim, J., Frankel, M. R., Augemberg, K., Prince, S.
- Evaluating Efficiency and Effectiveness of Cell Phone Samples; 2008; Sen, S., Zmud, J., Arce, C.
- The “Professional Respondent” Problem in Web Surveys; 2008; Rivers, D.
- Computing Metrics for Online Panels; 2008; Callegaro, M., DiSogra, C.
- Internet Surveys: Can Statistical Adjustments Eliminate Coverage Bias?; 2008; Dever, J. A., Rafferty, A., Valliant, R. L.
- Longitudinal tracking of voter perceptions during the 2007-2008 election cycle using a probability-based...; 2008; Tompson, T., Lawrence, M., Subias, S.
- Panel Surveys; 2008; Sikkel, D., Hoogendoorn, A.
- Does experience and sample attrition affect response behavior in internet panels? ; 2008; Binswanger, J., Schunk, D.
- New survey strategies for radio: the RFM’s “Ouvinte Mais” case study; 2008; Cordeiro, P.
- Assessing the Accuracy of Online Panel Research: A Toolkit for Decision Makers; 2008; Graham, P.
- Key Issues in Research Accuracy: Sources of bias and error in online research; 2008; Dennis, J. M., Callegaro, M.
- Making Quality Real: Delivering on a Promise of the Best Service and Online Survey Sample ; 2008; Dennis, J. M.
- Using global online panels; 2008; Pearson, C., Smith, E., Ridlen, R., Zhang, H., Cooper, A
- The quest for on-line quality research; 2008; Rhall, T., Fine, B.
- 10 Years of Meinungsplatz.de: Success in the Collection of Data for Targeted Audiences, Such as the...; 2008; Weyergraf, O.
- Panel Quality: Tips to Achieve Pure Sample, from Recruitment to Long-Term Engagement; 2007; Fitzgerald, D.
- What is Sexual Harassment? It Depends on Who Asks! Framing Effects on Survey Responses; 2007; Galesic, M., Tourangeau, R.
- The impact of cookie deletion on the accuracy of site-server and ad-server metrics: An empirical comScore...; 2007; Abraham, Ma., Meierhoefer, C., Lipsman, A.
- The case for publishing (some) online polls; 2007; Taylor, H.
- Sampling for web surveys; 2007; Rivers, D.
- Reconstructing childhood health histories using internet panels; 2007; Smith, J. P.